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ARI  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&D  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recom¬ 
mendations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  Training  and  Simulation  Technical  Area  (Simulation  Systems  Design 
Team)  of  the  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  performs  research  and  development  in  areas  that  inolude  training  simu¬ 
lation  with  applicability  to  military  training.  Of  speoial  Interest  is  re¬ 
search  in  the  area  of  training  device  design  and  use  requirements.  Adequate 
decision  support  for  determining  these  training  system  designs  is  not  cur¬ 
rently  available. 

This  report  provides  an  overview  of  recently  completed  and  planned  re¬ 
search  efforts  designed  to  provide  training  system  design  decision  support. 
It  may  be  used  by  research  program  planners  to  help  coordinate  future  ef¬ 
forts  in  this  area. 
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NEW  DIRECTIONS  IN  TRAINING  SIMULATION  RESEARCH:  GENERATING,  HANDLING, 
AND  DELIVERING  EMPIRICALLY  BASED  DECISION  SUPPORT 


EXECUTIVE  SUMMARY 


Requirements: 

To  provide  an  overview  of  recently  completed,  ongoing,  and  planned  re¬ 
search  to  develop  a  decision  support  system  for  the  Army  Training  Community. 
Such  a  system  should  provide  empirically  based  guidance  on  the  design  and 
use  of  training  devices  and  simulators. 


Procedure: 

The  paper  is  organized  into  two  main  sections  which  describe:  (1)  the 
generation  of  new  empirical  data  on  training  device  design  and  use  and 
(2)  how  these  data  will  be  handled  to  provide  decision  support.  Each  of 
these  sections  is  further  subdivided  to  describe  specific  research  efforts. 


Findings: 

New  empirical  data  on  training  device  design  and  use  are  being  gener¬ 
ated  with  increasing  frequency.  A  method  for  organizing  these  data  and  de¬ 
livering  empirically  based  decision  support  is  being  developed.  The  method 
involves  the  use  of  data  base  and  expert  system  techniques. 


Utilization  of  Findings: 

This  report  may  be  used  by  researchers  in  planning  for  research  on 
training  system  issues  and  will  facilitate  coordination  of  these  future 
research  efforts.  It  does  this  by  providing  a  brief  overview  of  recently 
completed,  ongoing,  and  planned  research  on  training  device  design  and  use. 
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NEW  DIRECTIONS  IN  TRAINING  SIMULATION  RESEARCH: 
GENERATING ,  HANDLING , -AND  DELIVERING 
-EMPIRICALLY  BASED  DECISION  SUPPORT1 


Introduction 

The  training  device  research  program  of  the  Army  Research  Institute  (ARI) 

Training  and  Simulation  Technical  Area  seeks  to  provide  empirically  based 
guidance  for  choosing  the  appropriate  training  device  configuration  and  train¬ 
ing  method  for  specific  Army  training  applications.  In  these  applications  the 

appropriate  training  method  may  require  the  use  of  a  training  device,  some  < 

other  type  of  training  simulation,  or  no  simulation  at  all.  It  is  our  belief 

that  the  most  cost  and  training  effective  training  program  may  be  designed  if 

well  established  psychological  rules  are  used  in  conjunction  with  empirically 

derived  training  methods  and  training  device  designs.  It  is  also  our  belief 

that  empirical  data  may  only  provide  useful  guidance  if  it  is  available  in  a 

form  tnat  is  both  well  organized  and  accessible  to  the  Army  training  community. 

The  purpose  of  this  paper  is  to  describe  the  recently  completed,  ongoing, 
and  planned  efforts  to  generate,  handle,  and  deliver  empirically  based  train¬ 
ing  design  guidance.  This  portion  of  our  training  device  research  program  is 
therefore  divided  into  two  main  thrusts:  (1)  the  development  of  empirical 
data  on  training  device  design  and  use,  and  (2)  the  organization  of  these 
data  Into  an  accessible  and  useable  form  as  decision  support. 

The  amount  of  technical  information  about  training  methods  is  increasing 
every  day.  Most  of  this  information  is  found  in  journal  articles,  technical 
reports,  and  handbooks  that  are  not  easily  available  to  training  design  de¬ 
cision  makers.  Computers  allow  us  to  acquire,  distribute,  and  manipulate 

both  old  and  new  information  in  ways  never  before  possible.  While  computers  i 

help,  the  organization  of  information  into  useable  knowledge  requires  time  . 

and  training.  As  a  result,  a  bottleneck  exists  between  the  technology  that  ' 

handles  information  and  the  humans  who  organize  that  information  into  useable 
knowledge,  or  use  that  information  in  the  application  of  knowledge  to  a  spe¬ 
cific  problem  domain.  The  problem  domain  in  our  research  program  is  training 
and  simulation. 

Training  simulation  may  refer  to  a  wide  variety  of  training  methods. 

Kinkade  and  Wheaton  (1972)  define  a  training  device  as  "any  arrangement  of 
equipment,  apparatus,  or  materials  which  provides  conditions  that  help  train¬ 
ees  learn  a  task."  Furthermore,  "training  devices  include  two  dimensional 
displays  (i.e.,  textual,  symbolic,  or  pictorial  material)  and  real  or  simu¬ 
lated  three-dimensional  apparatus"  (Kinkade  and  Wheaton,  1972,  p.  670).  It 
is  our  view  that  all  training  devices  are  simulations  but  not  all  simulations 
are  used  exclusively  for  training  (see  figure  1).  The  word  simulation  re¬ 
fers  to  "the  Imitative  representation  of  the  functioning  of  one  system  or 
process  by  means  of  another"  (Webster's  New  Collegiate  Dictionary,  1979). 


TMajor  portions  of  this  paper  were  presented  at  the  Fifth  Interservice/ 
Industry  Training  Equipment  Conference,  Washington,  D.C.,  November  1 4— 16, 
1983. 
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According  to  this  broad  definition  of  simulation  there  can  be  many  varieties 
of  simulations  besides  those  used  for  training  purposes.  Simulations  may  be 
used  for  engineering  analysis,  as  in  the  case  where  a  wind  tunnel  and  a  re¬ 
duced  size  model  is  used  to  design  aircraft  or  automobiles.  Computer  simu¬ 
lations  are  used  to  forecast  the  effects  of  various  changes  in  the  economy. 
Training  simulation  is  another  use  of  imitative  representation.  In  this 
case,  the  desired  behavior  to  be  trained  can  be  simulated  in  the  classroom 
and  students  may  gain  experience  before  they  must  deal  with  real  world  con¬ 
straints.  These  training  simulations  may  or  may  not  use  training  devices. 


Figure  1.  The  relationship  of  training  devices  and  simulations. 


Figure  1  is  a  graphic  representation  of  our  view  of  the  relationship  of 
training  devices  and  simulations.  In  this  view,  training  devices  are  a  sub¬ 
set  of  all  possible  types  of  simulations.  Within  the  subset  of  all  training 
devices,  a  specific  training  application  may  require  a  unique  training  device. 
There  are  many  factors  that  enter  into  the  decisions  about  the  specification 
of  training  device  characteristics.  Some  general  information  about  a  few 
gross  factors  and  instructional  features  that  may  be  used  in  training  device 
applications  is  provided  in  the  Appendix. 

The  rest  of  this  paper  is  organized  into  two  main  sections,  each  deal¬ 
ing  with  a  specific  focus  of  our  portion  of  the  research  program.  The  first 
section  describes  ongoing  and  planned  research  designed  to  provide  data  which 
answer  fundamental  questions  about  training  device  design  and  use.  The  sec¬ 
ond  section  of  the  paper  discusses  ongoing  and  planned  efforts  to  organize 
and  interpret  empirical  data,  and  deliver  empirically  based  decision  support. 
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Generating  New  Empirical  Data 


1 


A  major  portion  of  our  simulation  research  program  has  centered  on 
field  and  laboratory  research.  This  section  provides  a  brief  description 
of  these  research  efforts.  The  multiyear  field  research  effort  described 
in  this  section  consisted  of  experimental  and  analytical  evaluations  of 
training  device  concepts.  The  evaluations  were  conducted  in  support  of 
the  Army  Maintenance  Training  and  Evaluation  Simulation  System  (AMTESS) 
program.  The  laboratory  research  described  in  this  section  consists  of 
planned  research  using  the  training  devices  built  during  the  earlier 
phases  of  the  AMTESS  program,  and  the  ongoing  basic  research  in  simulator 
fidelity.  Since  efforts  in  both  the  field  and  the  laboratory  are  part  of 
the  AMTESS  program,  a  brief  description  of  the  AMTESS  program  is  provided 
next. 

The  Army's  Project  Manager  for  Training  Devices  (PM  TRADE)  initiated 
the  AMTESS  program  in  1978.  PM  TRADE  has  been  the  lead  agency  and  has 
been  supported  by  a  Joint  Working  Group  (JWG)  consisting  of  representa¬ 
tives  from  ARI ,  Training  and  Doctrine  Command  (TRADOC),  Army  Schools,  and 
other  Army  agencies.  ARI's  role  in  the  JWG  was  to  provide  technical 
expertise  on  training  effectiveness  assessment  and  front  end  analysis  of 
the  AMTESS  devices.  ARI  also  conducted  human  factors  assessments  of  the 
AMTESS  devices,  analytic  studies  of  the  device  procurement  process,  and 
developed  plans  to  conduct  laboratory  experiments  using  the  AMTESS 
devices. 

The  three  main  objectives  of  AMTESS  are  to  provide  the  Army  with: 

1)  A  hardware  model  for  development  of  generic,  modular  maintenance 
trainers.  These  trainers  would  consist  of  a  generic  core  module 
(instructor  and  student  stations  and  controlling  micro  computer)  and  a 
task  specific  3D  module  for  hands-on  practice. 

2)  Front  end  analysis  procedures  which  would  provide  guidance  for 
conducting  task,  training  and  fidelity  requirements  analyses. 

3)  Guidance  for  the  broad  application  of  AMTESS  concepts. 

The  first  phase  of  the  AMTESS  evaluation  consisted  of:  a  training 
effectiveness  assessment  (TEA),  a  cost  analysis,  an  analysis  of  authoring 
capabilities  of  the  AMTESS  device  and  an  engineering  analysis.  ARI's 
major  role  was  to  conduct  the  TEA.  The  objectives  of  this  assessment  were 
to:  develop  insights  into  effective  device  design,  develop  specifications 
for  the  AMTESS  hardware  model,  and  to  develop  a  model  for  device  acquisi¬ 
tion  . 
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that  can  be  applied  to  generate  very  specific  recommendations  in  that 
domain  (e.g.,  the  design  of  a  specific  training  devices).  We  are  using 
these  expert  system  techniques  to  develop  the  prototype  training  device 
design  decision  support  system. 

Decison  Support.  In  our  expert  system,  now  in  development,  user 
input  and  internally  stored  information  about  the  types  of  tasks  to  be 
trained  is  combined  by  the  system  to  generate  a  set  of  recommendations. 

The  system  does  this  by  questioning  the  user  about  the  types  of  task  to  be 
trained  and  extracts  information  from  the  user,  (in  the  future  it  should 
also  be  able  to  extract  information  from  other  computer  files).  These 
questions  are  about  the  kind  of  training  format  that  should  be  followed, 
or  some  of  the  kinds  of  features  that  should  be  incorporated  in  the 
training  device  in  order  to  fit  the  training  program.  The  system  rules  use 
this  information  to  create  new  information  about  the  training  needs,  or 
the  adjunct  features  required  to  provide  adequate  support  by  the  training 
device.  The  generation  of  these  recommendations  is  based  on  the  structured 
knowledge  in  the  data  base  that  provided  the  rules  of  the  expert  system. 
Therefore  the  system  provides  an  information  addition  based  on  previously 
established  knowledge,  and  provides  guidance  or  training  device  design 
recommendations. 

This  ongoing  research  effort  is  something  more  than  a  decision 
support  and  data  base  effort.  The  organized  information,  which  is  empiri¬ 
cally  based  information  from  reported  research,  provides  a  structured 
approach  to  training  device  design  and  use.  This  knowledge  of  structured 
relations  among  the  various  factors,  as  represented  by  the  rules  written 
into  the  expert  system,  can  also  be  used  to  guide  future  research  in  simu¬ 
lation  issues.  When  the  system  is  used,  each  application  becomes  an  op¬ 
portunity  to  collect  more  data  about  the  training  effectiveness  of  the 
simulator  and  the  training  program.  For  example,  when  the  system  recom¬ 
mends  some  specific  level  of  fidelity  and  amount  of  practice  in  training  a 
task  some  level  of  proficiency  is  expected  to  develop.  If  the  level  of 
proficiency  is  much  higher  or  much  lower  than  expected,  a  deficiency  is 
identified  in  either  the  implementation  or  the  system  recommendation.  If 
the  deficiency  is  in  the  decision  system,  research  would  be  indicated  on 
the  complex  interaction  of  factors  that  produced  the  unexpected  level  of 
proficiency.  In  this  way  the  development  and  use  of  this  expert  system 
can  do  more  than  provide  information  and  recommendations  about  training 
device  design  and  use  to  the  training  device  developer. 

An  added  benefit  is  that  while  building  the  system  we  are  also  defin¬ 
ing  the  gaps  in  our  knowledge  about  training  device  design  and  use.  To  a 
large  extent  there  are  no  general  experts  in  training  device  design,  and 
thus  the  expertise  that  we  will  call  upon  is  application  specific.  As  an 
example,  we  consult  an  expert  in  deisel  mechanic  training  about  a 
diesel  mechanic  training  device.  Our  task  is  to  use  the  knowledge  of  the 
specialist  to  create  general  rules  for  use  in  many  different  training 
device  designs  and  applications.  In  addition,  much  of  the  research  that 
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nomenclature).  Our  goal  is  to  develop  decision  support  rules  based  on 
these  general  conclusions  rather  than  rules  based  on  a  single  research 
effort. 


Delphi  Analysis.  Delphi  analysis  (Linstone  4  Turoff,  1975)  is  another 
technique  that  we  expect  to  use  extensively  in  extracting  knowledge  for 
the  decision  support  system  from  the  data  base.  It  is  a  semi-structured 
procedure  in  which  subject  matter  experts  collaborate  to  achieve  consensus 
on  an  issue.  In  our  situation  we  will  use  in-house  expertise  to  extract, 
review,  and  evaluate  the  verbal  information  in  the  data  base  (e.g.,  ab¬ 
stract,  reviewers  comments,  etc.).  The  goal  will  be  to  develop  a  rule 
that  accurately  states  the  relation  between  a  task  or  training  parameter, 
a  training  device  factor,  and  other  training  system  factors.  The  experts 
will  sort  through  the  data  base  for  related  articles  and  then  meet  several 
times  to  evaluate  those  articles.  At  some  point  a  consensus  about  the 
knowledge  contained  in  those  articles  will  be  reduced  to  rule  form  for 
inclusion  in  the  decision  support  system. 

Structured  Interviews.  Structured  interviews  of  trainers  and  training 
device  developers  will  also  be  used  to  acquire  relevant  knowledge  for  the 
decision  support  system.  We  plan  to  adapt  versions  of  the  structured 
interview  format  used  in  Criswell,  et.  al.,  (1983)  and  Woelfel,  et.  al. 
(1984)  as  a  starting  point  in  this  process.  The  general  plan  at  this  time 
is  to  present  prospective  users  with  the  demonstration  system  and  obtain 
their  evaluations  of  the  recommendations  produced.  The  trainers  and 
training  developers  will  use  their  actual  training  situations  as  input  to 
the  the  system.  The  judged  differences  between  the  system  recommendations 
and  what  the  trainers  and  training  developers  consider  to  be  optimal  will 
hopefully  lead  to  new  and  improved  rules  for  use  in  the  system.  Also 
their  feedback  about  the  decision  support  systems  operation  should  be 
useful  in  improving  the  system's  user  interface. 

Delivering  the  Information 

Usually  when  information  is  organized  to  provide  guidance  to  a  spe¬ 
cialist  it  is  produced  in  the  format  of  a  guidebook.  As  mentioned  before 
computers  are  allowing  new  solutions  to  old  problems  and  we  are  developing 
new  methods  for  providing  guidance,  using  new  techniques.  The  techniques 
are  generally  called  expert  systems,  and  are  a  branch  or  subset  of 
Artifical  Intelligence  research.  The  decision  support  system  acquires 
information  from  the  user  and  combines  this  information  with  that  already 
contained  in  the  system  to  generate  recommendations. 

Artificial  Intelligence.  Artificial  Intelligence  refers  to 
methodologies  based  in  the  study  of  human  information  handling  patterns. 
The  approach  we  are  using  is  a  subset  called  expert  system  techniques. 

This  involves  organizing  information  in  heuristics  (Smith,  1983)  which  are 
formal  rules  in  a  computer  program.  These  rules  represent  general  knowl¬ 
edge  about  a  problem  domain  (in  our  situation,  simulators  and  training) 
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to  1982.  The  reports  and  articles  were  selected  on  the  basis  of  their 
relevance  to  simulator  design,  simulator  evaluation,  and  simulator  effec¬ 
tiveness  in  training.  Plans  are  currently  being  developed  to  restucture 
the  data  of  text  files  Into  a  more  usable  on-line  data  base  format.  In 
general  the  major  points  described  above  and  some  of  the  sub  items  will 
be  used  as  records  in  the  data  base.  The  data  base  can  then  be  used  in 
conjunction  with  various  techniques  to  extract  and  organize  the  knowledge 
for  use  in  decision  support  system  and  provide  user  support  for  the  de>  • 
sion  system. 

In  order  for  the  organized  information  in  the  data  base  to  be  useful 
to  the  decision  support  system  user,  it  must  be  easy  to  access  the 
empirical  referents  which  support  the  explicit  recommendations  that  the 
system  generates.  Our  scenario  assumes  that  there  may  be  some  topic  on 
which  the  user  wants  additional  information  about  the  generated 
recommendation  that  is  a  result  of  several  rule  applications  in  the 
decision  system  (for  example,  the  short  summary  of  an  article  that  is  the 
basis  of  a  rule  in  the  decision  system).  The  system  of  heuristics  or  rules 
used  in  the  decision  support  system  will  be  referenced  to  specific  au¬ 
thors,  keywords,  subject  matter  expert  opinions,  and  articles.  In  this 
way  the  user  can  directly  consult  the  source  from  which  the  rule  (see 
below)  was  drawn  for  the  recommendation.  In  other  words,  the  user  of  the 
decision  support  system  can  evaluate  the  generated  recommendation  by 
looking  at  the  empirical  information  that  supports  the  decision  system. 

Using  the  Information 

We  are  investigating  several  methodologies  for  organizing  and 
restructuring  the  information  for  use  in  the  decision  support  system.  Each 
methodology  has  strengths  and  weaknesses,  and  we  are  applying  three 
techniques  to  ensure  against  loss  of  important  knowledge.  These 
methodologies  are  going  to  be  used  to  extract  and  organize  knowledge 
about  the  relationships  and  factors  involved  in  the  design  of -training 
devices  and  their  use  in  training  programs.  One  methodology  we  plan  to 
use  is  meta-analysis,  which  will  draw  on  key  statistical  data  in  the  data 
base.  A  second  methodology  we  plan  to  apply  will  be  a  delphi  process 
which  will  draw  on  the  information  in  the  abstract,  the  short  summary,  and 
the  commentary  in  the  data  base.  A  third  methodology  we  plan  to  use  consists 
of  structured  interviews  of  trainers  and  training  developers  that  draws 
upon  their  expertise  to  generate  information  not  available  in  the 
literature. 

Meta-Analysi3.  Meta-analysis  is  a  general  type  of  statistical  analysis 
that  treats  individual  experiments  as  pieces  of  data  in  a  single 
experiment.  The  summary  data  from  each  experiment  are  used  to  determine  the 
overall  effect  size  of  the  major  factors  concerned  in  all  of  the 
experiments  (Glass,  McGraw,  &  Smith,  1981).  This  type  of  analysis  is 
used  to  arrive  at  general  conclusions  about  the  relationships  among 
various  factors  (e.g.,  computer  assisted  instruction  and  retention  of 
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with  these  task  parameters  (e.g.,  instructor-student  ratio,  type  of  in¬ 
structional  strategies,  etc.).  Also,  in  order  to  evaluate  the  effective¬ 
ness  of  the  training  that  has  been  provided,  proficiency  in  the  trained 
task  must  somehow  be  measured.  How  to  relate  this  proficiency  measurement 
to  overall  Army  force  readiness  is  not  at  all  clear.  However,  without 
some  kind  of  relevant  task  performance  measure  there  is  no  way  to  compare 
the  outcome  of  different  training  programs,  or  make  comparisons  within  a 
single  training  program  using  different  kinds  of  training  equipment.  All 
of  these  points  must  be  considered  in  the  delivery  of  decision  support. 

Our  initial  effort  is  descriptive  and  primarily  serves  to  identify 
and  use  knowledge  about  training  factors  and  relationships  from  existing 
guidance  documents  (e.g.,  Kinkade  &  Wheaton,  1972).  The  effort  is  also 
allowing  us  to  evaluate  these  factors  and  relationships  in  terms  of 
importance  in  describing  and  predicting  simulation  based  training  program 
effectiveness.  In  addition  this  assessment  allows  the  identification  of 
research  areas  that  can  provide  information  that  would  improve  Army 
training.  When  research  on  these  issues  produces  new  empirical  data  they 
will  be  entered  into  the  data  base  on  training  information  that  is 
discussed  next. 


Data  Base  On  Training  Device  Information.  In  identifying  and  organizing 
the  training  device  factors  we  must  be  sure  that  existing  knowledge  is  not 
overlooked.  Therefore,  we  have  initiated  a  data  base  effort  that  we 
expect  to  develop  into  an  applied  research  support  tool  (Hays  A  Singer, 

1983)  and  which  will  provide  the  empirical  foundation  for  a  decision 
support  system  for  training  device  design  and  use. 

The  data  base  currently  consists  of  text  files  containing  data  from 
applications  oriented  research  in  simulator  fidelity  and  training 
effectiveness  issues.  The  information  is  condensed  from  the  empirical 
reports  and  journal  articles  through  the  use  of  a  structured  and 
extensively  annotated  abstract  format  (Ayres,  Hays,  Singer  &  Heinicke, 

198*0.  The  files  also  contain  information  from  theoretical  reports 
that  have  been  abstracted  in  the  same  way.  Each  abstract  is  organized 
according  to  the  same  9-point  format,  under  the  following  headings: 

(1)  Authors;  (2)  Title;  (3)  Source;  (4)  Topic  Keyword;  (5)  Short  Summary; 

(6)  Devices  discussed  or  studied  (7)  Institution;  (8)  Type  of  Article;  and 
(9)  Abstract.  These  9  headings  are  further  broken  down  into  subheadings 
and  filled  in  with  appropriate  information  (e.g.,  type  of  device,  subject 
characteristics,  key  data,  etc.)  if  available  in  the  original  document.  We 
will  be  extending  the  data  base  by  reviewing  and  adding  information  from 
more  recent  relevant  reports  and  articles.  The  current  version  of  the 
data  base  contains  149  annotated  abstracts  spanning  a  time  frame  from  1953 


Handling  Empirical  Data  and  Delivering 
Empirically  Based  Decision  Support 

The  data  generation  efforts  discussed  in  the  previous  section  are 
providing  large  amounts  of  data.  Those  efforts  are  one  example  of  the 
increase  in  research  into  training  and  simulation  issues.  In  addition  a 
substantial  body  of  information  has  been  accumulated  in  the  literature  in 
recent  years  (see  Ayres,  Hays,  Singer  &  Heinicke,  1981*).  At  the  Army 
Research  Institute,  our  team  has  begun  to  study  how  to  organize  and  handle 
the  vast  and  growing  amount  of  training  and  simulation  information 
(Hays  &  Singer,  1983).  Our  goal  is  to  demonstrate  the  feasibility  of 
providing  empirically  based  training  device  requirements  decision  support 
to  Army  training  and  procurement  personnel. 

In  order  to  provide  useful  decision  support  for  the  Army  training 
community,  three  stages  are  necessary.  First,  a  structured  approach  for 
selecting  and  organizing  appropriate  research  data  and  information  from 
the  literature  is  required.  Second,  techniques  for  using  the  information 
in  developing  decision  rules  or  guidance  must  be  applied.  Third,  an 
adequate  means  for  delivering  the  decision  support  to  the  user  must  be 
developed. 

Organizing  the  Information 

The  exponential  growth  of  information,  combined  with  the  ever 
increasing  technology,  has  out-stripped  our  normal  methods  of  developing 
experts.  The  overload  is  exacerbated  by  the  vast  range  of  information 
sources  that  must  be  monitored  in  order  to  maintain  expertise  in  training 
and  simulation.  The  training  developer  doesn't  have  the  time  to 
continually  sort  through  the  huge  volume  of  information  generated, 
integrate  that  information  into  knowledge  and  all  while  still  performing 
normal  duties.  As  a  result,  training  developers  have  a  difficult  task  in 
specifying  simulators.  In  addition,  it  is  not  easy  to  determine  the 
optimal  integration  of  simulators  into  training  programs.  Our  approach  is 
to  first  identify  and  organize  the  various  factors  involving  simulators 
and  training  devices  and  their  use  in  training  programs  (Hays  &  Singer, 
1983).  Once  the  issues  have  been  identified  and  organized  the  empirical 
data  on  those  issues  must  be  accumulated  in  a  data  base.  The  data  base 
then  provides  the  empirical  justification  for  the  rules  and  recommenda¬ 
tions  in  a  decision  support  system. 

Training  Device  Factors  In  Training  Programs.  One  major  set  of  fac¬ 
tors  and  relations  that  we  are  addressing  in  the  development  of  decision 
support  are  the  task  parameters  in  the  training  program.  The 
to-be-trained  tasks  provide  one  important  initial  decision  constraint  on 
whether  to  use  a  training  device,  what  the  training  device  should  be  like, 
and  how  to  use  that  training  device  to  best  effect.  There  are  additional 
factors  and  relationships  in  the  training  context  itself  that  interact 
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The  experiments  offer  a  unique  opportunity  to  generate  programmatic 
data  on  fidelity  effects.  They  are  supported  by  a  generic  device  specif¬ 
ically  constructed  for  this  effort.  The  device  consists  of  a  series  of 
electro-mechanical  relays  which  control  a  set  of  output  devices  (e.g., 
pumps,  lights,  TV  monitor,  etc.)  and  into  which  malfunctions  may  be  in¬ 
serted.  Several  degraded  simulators  of  the  generic  device  have  also  been 
constructed  and  will  serve  as  instruments  for  training  subjects  in  one  of 
the  nine  cells  of  the  matrix  in  Figure  5.  These  experiments  will  be  com¬ 
pleted  during  the  summer  of  1985. 

Summary  of  Data  Generation  Efforts.  This  major  portion  of  ARI ' s 
simulation  research  program  is  generating  new  empirical  data  that  can 
support  guidance  for  the  Army  training  community.  The  sources  of  these 
are:  1)  the  AMTESS  field  evaluations,  2)  the  AMTESS  analytical  research, 

3)  the  planned  AMTESS  laboratory  research  program,  and  4)  the  basic  research 
program  on  simulator  fidelity.  The  generation  of  these  new  empirical  data 
create  a  new  problem:  how  to  handle  the  large  amounts  of  information  in 
an  efficient  and  useful  manner.  The  next  section  discusses  new  methods 
for  dealing  with  this  information  in  order  to  provide  useful  decision 
support. 
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PHYSICAL  SIMILARITY 


Figure  5.  Nine  cell  matrix  for  training  simulator  fidelity  experiments. 


Table  2 


Variables  Which  Interact  with  Fidelitj 


Task  Type 

-  Operations 

-  Maintenance 

-  Others 

Task  Difficulty 

Specific  Skills  Required  by  Task 

-  Motor 

-  Perceptual 

-  Cognitive 

-  Others 

Trainee  Sophistication 

-  Novice 

•  Intermediate 

-  Expert 


Stage  of  Training 

-  Introduction 

-  Procedural  Training 

-  Familiarization  Training 

-  Skill  Training 

-  Transition  Training 

Training  Context 

-  Institutional 

-  Field 

Incorporation  of  Device  into  POI 

User  Acceptance 

-  Instructors 

-  Students 

Use  of  Instructional  Features 


Planned  AMTESS  Laboratory  Research,  Although  large  amounts  of  qualitative, 
quantitative,  and  analytic  data  have  been  generated  from  field  studies  of 
the  AMTESS  devices,  many  questions  still  remain  unanswered.  In  order  to 
determine  the  training  effectiveness  of  specific  device  features,  we  need 
to  conduct  better  controlled  experiments.  With  this  goal  in  mind,  ARI  and 
PM  TRADE  have  moved  the  AMTESS  breadboards  to  George  Mason  University 
(GMU).  Experiments  at  GMU  will  focus  on  the  training  effects  of  several 
training  system  variables  including:  task  type,  trainee  characteristics, 
Instructional  strategies,  and  specific  training  device  features.  These 
experiments  will  be  conducted  by  private  contractors,  by  GMU  faculty  and 
students,  and  by  ARI  personnel.  They  are  expected  to  provide  data  which 
can  be  used  as  guidance  for  training  device  developers  and  instructional 
designers. 

Simulator  Fidelity  Research.  The  degree  to  which  a  training  device  or 
simulator  resembles  the  operational  equipment  is  referred  to  as  simulator 
fidelity.  For  many  years,  researchers  have  attempted  to  determine  the 
optimal  level  of  fidelity  for  cost  effective  training.  ARI  has 
established  a  research  program  to  systematically  accumulate  the  data 
needed  to  make  fidelity  tradeoff  decisions. 

The  first  step  in  this  effort  was  to  develop  a  consistent  and  useful 
definition  of  simulator  fidelity.  After  a  review  of  the  literature  (Hays, 
1980)  and  a  fidelity  workshop  which  brought  together  researchers  from 
academia,  industry  and  the  government  (Hays,  1981),  the  following  defini¬ 
tion  of  simulator  fidelity  was  adopted  to  guide  the  research  program. 

Training  Simulator  Fidelity  is  the  degree  of  similarity 
between  the  training  simulator  and  the  equipment  which 
is  simulated.  It  is  a  two  dimensional  measurement  of 
this  similarity  in  terms  of: 

-  the  physical  characteristics  of  the  training  simulator 

-  the  functional  characteristics  (i.e.,  the  informational 
or  stimulus  and  response  options)  of  the  simulated 
equipment  (Hays,  1981,  p.  70). 

This  definition  was  then  implemented  in  a  basic  experimental  design  for 
exploratory  research  on  simulator  fidelity.  Figure  5  shows  a  nine  cell 
matrix  incorporating  three  ordinal  levels  of  each  fidelity  dimension.  The 
first  experiment  to  use  this  design  investigated  the  effects  of  simulator 
fidelity  for  a  mechanical  adjustment  task.  The  rationale  for  this 
experiment  is  reported  in  Baum,  et  al.,  (1982a)  and  the  results  of  the 
experiment  are  reported  in  Baum,  et  al.,  (1982b). 

A  second,  more  ambitious  effort  is  currently  underway.  It  consists  of 
a  series  of  fidelity  experiments  dealing  with  an  electro-mechanical 
troubleshooting  task.  A  description  of  this  effort  is  provided  by  Allen 
and  Hays  (1983).  The  first  experiment  in  this  series  will  look  solely  at 
the  fidelity  variable.  Subsequent  experiments  will  begin  investigations 
of  variables  believed  to  interact  with  fidelity  to  produce  different 
levels  of  training  effectiveness.  Table  2  provides  a  list  of  some  of 
these  suspected  interactive  variables. 
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malfunctions  that  would  damage  real  equipment  in  the  automotive  mode). 
However,  respondents  criticized  the  low  reliability  of  the  simulators  and 
the  inappropriate  fidelity  levels  for  certain  tasks.  For  certain  tasks 
the  fidelity  level  was  regarded  as  too  low  because  critical  components 
were  not  included  in  the  simulators.  On  the  other  hand,  for  other  tasks 
the  fidelity  level  was  regarded  as  too  high  for  familiarization  training. 
Future  simulator  designers  will  either  have  to  make  fidelity-task  type 
tradeoffs  to  insure  the  most  training  effective  overall  fidelity  level  or 
incorporate  modules  with  increasing  levels  of  fidelity  as  specific  task 
training  requires  them.  Additional  research  is  needed  to  provide  more 
detailed  specifications  for  these  fidelity  decisions.  The  results  of  both 
the  quantitative  and  qualitative  evaluations  are  documented  in  Unger,  et 
al.,  1984  (3  Volumes). 

AMTESS  Analytical  Research,  other  analytic  efforts  have  been  under¬ 
taken  to  provide  additional  data  on  the  AMTESS  devices  and  the  AMTESS 
program.  Criswell,  et.  al.,  (1983)  documented  the  history  of  the  AMTESS 
program  and  collected  opinion  data  on  specific  features  of  the  AMTESS 
devices.  The  AMTESS  history  highlighted  several  problems  which  arose  dur¬ 
ing  AMTESS  device  development,  acquisition  and  testing.  Most  of  these 
problems  centered  on  the  need  for  (1)  more  frequent,  more  precise  communi¬ 
cations,  (2)  clearer  definitions  of  the  explicit  responsibilities  of  each 
agency,  both  governement  and  contractor,  (3)  more  explicit  mechanisms  for 
quality  control  of  the  devices,  (4)  greater  anticipation  of  disruptive 
contingencies,  and  (5)  the  need  for  high-level  administrative  and  finan¬ 
cial  resources  appropriate  to  the  responsibilities  imposed  on  program 
personnel.  Interviewers  felt  that  many  device  features  were  based  on 
sound  instructional  concepts.  Implementation  problems,  however,  especially 
disrepair,  plagued  both  devices.  Some  of  the  most  valued  features  of  the 
devices  were  their  high  fidelity  3D  modules  and  the  comprehensive  student 
performance  records.  The  opinion  data  on  device  features  will  be  valuable 
to  future  device  designers  and  the  accumulation  of  "lessons 
learned"  during  the  AMTESS  project  can  help  in  planning  future  efforts 
like  AMTESS. 

Woelfel,  et  al.,(1984)  evaluated  the  front-end  analysis  (FEA) 
procedures  used  by  the  AMTESS  hardware  contractors  and  by  the  government 
during  the  early  phases  of  the  project.  They  cataloged  and  evaluated  the 
FEA  procedures  used  in  the  AMTESS  project  and  recommended  FEA  guidelines 
to  facilitate  future  FEA  activities  in  AMTESS  type  programs.  The  review 
of  FEA  activities  was  guided  by  four  criteria:  (1)  comprehensiveness  of 
performance,  (2)  clarity  of  requirements,  (3)  effectiveness  of  coordina¬ 
tion,  and  (4)  reliability  and  maintainability  (RAM)  analysis,  (5)  organi¬ 
zation  analysis,  (6)  human  factors  analysis,  and  (7)  person  analysis. 

Laboratory  Research 

In  addition  to  the  field  research  described  above,  several  planned  or 
ongoing  laboratory  research  projects  will  provide  additional  empirical 
data  on  training  device  design  and  use.  One  of  these  efforts  involves  the 
planned  use  of  the  AMTESS  devices.  The  second  effort  is  an  ongoing  basic 
research  program  on  the  question  of  simulator  fidelity. 
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A  summary  of  the  results  of  the  evaluation  of  the  electronics  mode  of 
the  Seville/Burtek  device  is  presented  in  Figure  4.  For  these  electronics 
tasks,  almost  70S  of  the  comparisons  between  Seville/Burtek  and  conven¬ 
tionally  trained  groups  showed  no  performance  differences.  Conventionally 
trained  students  did  perform  several  tasks  faster  than  simulator  trained 
students  but  on  a  small  number  of  tasks  simulator  trained  student  perfor¬ 
mance  exceeded  that  of  conventionally  trained  students. 


MOS  24C10 


69%  OF  COMPARISONS  INDICATE  NO  SIGNIFICANT 
DIFFERENCES 


Figure  4.  Summary  of  comparisons  of  conventional  training  to  training  with 
the  Seville/Burtek  device.  Ft.  Bliss,  Texas. 


As  mentioned  above,  it  was  not  possible  to  obtain  a  comparable 
conventionally  trained  group  to  use  in  evaluating  the  Grumman  device  in  its 
electronics  mode.  Ten  subjects  were  trained  on  the  device,  however,  and 
were  able  to  perform  the  same  electronics  maintenance  tasks  as  those 
trained  on  the  Seville/Burtek  device.  In  fact,  no  differences  in 
performance  could  be  found  between  groups  trained  on  either  version  of 
AHTESS  in  the  electronics  mode  when  tested  on  the  actual  equipment. 

Opinions  of  Instructors,  course  developers,  and  students  about  the 
AMTESS  devices  were  also  collected  during  the  evaluations.  These  opinions 
were  mostly  positive.  The  devices  were  viewed  as  useful  training  media 
especially  for  tasks  which  could  not  be  trained  on  the  operational 
equipment  (e.g.,  high  voltage  troubleshooting  in  the  electronics  mode  or 


'  \ 


8 


MOS  63D30 


Figure  3 


78%  OF  COMPARISONS  INDICATE 
NO  SIGNIFICANT  DIFFERENCES 


CONVENTIONAL 


MOS  63H30 


78%  OF  COMPARISONS  INDICATE 
NO  SIGNIFICANT  DIFFERENCES 


Sumnary  of  comparisons  of  conventional  training  to  training  with 
the  Grumman  device,  Aberdeen  Proving  Ground,  Maryland. 
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MOS  63B30 


Figure  2 


71%  OF  COMPARISONS  INDICATE 
NO  SIGNIFICANT  DIFFERENCES 


MOS  63W10 


48%  OF  COMPARISONS  INDICATE 
NO  SIGNIFICANT  DIFFERENCES 


Summary  of  comparisons  of  conventional  training  to  training  with 
the  Seville/Burtek  device,  Aberdeen  Proving  Ground,  Maryland. 
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evaluation  provides  expert  opinion  data  on  device  features  and  the  procedures 
used  in  AMTESS  device  development. 

AMTESS  Field  Evaluations.  Trainees  were  taught  either  conventionally 
or  on  an  AM1ESS  device  and  their  performance  was  tested  with  hands-on  ex¬ 
ercises  on  operational  equipment.  Since  the  standard  school  performance 
tests  generate  gross  overall  indices  of  trainers  performance  and  a  more 
detailed  evaluation  was  required,  new  performance  tests  were  developed. 

These  tests  consisted  of  detailed  step  by  step  measures  of  performance  on 
selected  criterion  tasks.  The  performance  of  the  trainees  was  evaluated  in 
terms  of  time,  accuracy,  and  required  instructor  interventions. 

The  Seville/Burtek  device,  in  its  automotive  configuration,  was  tested 
in  two  MOSs:  63B30  (Organizational  Maintenance  Supervisor)and  63W10  (Di¬ 
rect  Support  Vehicle  Repairman).  Its  electronics  version,  was  tested  in 
MOS  24C10  (Hawk  Missile  Firing  Section  Mechanic).  The  Grumman  device,  in 
its  automotive  configuration  was  also  tested  in  two  MOSs:  63D30  (Self- 
propelled  Field  Artillery  Mechanic)  and  63H30  (Direct  Support  Maintenance 
Supervisor).  The  Grumman  electronics  configuration  was  tested  in  MOS 
24C10  (Hawk  Missile  Firing  Section  Mechanic).  However,  the  Grumman  elec¬ 
tronic  configuration  was  not  compatible  with  conventional  training  and 
therefore,  it  was  not  possible  to  obtain  a  comparison  group.  Only  the 
performance  of  students  trained  on  the  Grumman  AMTESS  electronics  device 
was  assessed  without  comparison  to  a  conventional  training  group. 

Generally,  the  performance  of  AMTESS  trained  and  conventionally 
trained  students  was  equivalent.  All  students  passed  the  schools’ 
criteria  for  successful  accomplishment  of  the  tasks  trained  during  the 
evaluation.  Where  differences  did  exist,  in  general  the  conventionally 
trained  group  was  superior.  Most  of  these  differences  were  in  the  time  to 
complete  the  criterion  task  and  in  the  number  of  instructor  interventions 
required  for  the  trainee  to  complete  the  task.  In  most  cases,  however, 
the  AMTESS  trained  students  could  perform  the  task  but  required  some 
additional  orientation  on  the  operational  equipment  since  the  AMTESS 
devices  often  did  not  contain  critical  components  normally  found  on  the 
operational  equipment. 

Figure  2  is  a  summary  of  all  comparisons  on  the  three  dependent 
measures  at  Aberdeen  Proving  Ground  between  groups  trained  on  the 
Seville/Burtek  device  and  groups  trained  conventionally.  In  MOS  63W10,  an 
entry  level  MOS,  521  of  comparisons  showed  conventional  training  superior 
to  AMTESS  training.  This  figure  drops  to  only  29 %, however,  in  MOS  63B30, 
a  higher  skill  level  MOS.  Apparently,  trainees  with  more  experience  on 
the  operational  equipment  do  not  require  as  much  operational  equipment 
orientation  after  AMTESS  training  as  do  entry  level  trainees. 

A  similar  summary  of  all  comparisons  of  the  Grumman  device  and 
conventionally  trained  groups  in  the  automotive  area  is  provided  in  Figure  3. 
Here  we  see  that  in  both  MOS  63D30  and  63H30  almost  80%  of  the  comparisons 
showed  no  differences  between  training  conditions.  As  with  the  Seville 
Burtek  device,  differences  that  were  found  consisted  of  time  delays  and 
increased  frequency  of  instructor  interventions  for  the  Grumman  trained 
group.  Overall  ability  to  correctly  complete  the  criterion  tasks  did  not 
differ  between  training  conditions. 
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The  AMTESS  program  seeks  to  develop  and  test  a  model  for  generic  main¬ 
tenance  simulation  training  systems.  The  model  consists  of  two  components: 
The  core  component  and  a  task  specific  3D  modular  component.  The  core 
component  controls  the  device  and  includes  a  general  purpose  micro  proces¬ 
sor,  a  2D  visual  display  interface,  and  an  instructor  station.  The  3D 
modular  component  consists  of  either  a  simulation  of  a  complete  unit 
(e.g.,  an  engine)  or  simulations  of  component  units  (e.g.,  battery, 
starter  motor).  Table  1  is  a  summary  of  the  designs  of  two  AMTESS 
breadboard  devices  that  were  evaluated  at  Army  Schools.  A  breadboard 
device  is  an  early  engineering  development  tool  which  allows  formative 
evaluations  of  design  concepts  prior  to  the  prototype  phase.  One  AMTESS 
breadboard  device  was  developed  by  a  consortium  of  Seville  Research  and 
Burtek,  Inc.  and  the  other  was  developed  by  Grumman  Aerospace.  Both  ver¬ 
sions  followed  the  general  AMTESS  model  described  above.  The  versions 
differed  in  their  microprocessors,  their  2D  instructional  delivery  sys¬ 
tems,  the  tasks  they  were  designed  to  train  and  their  instructional  phi¬ 
losophies.  Each  AMTESS  breadboard  device  was  built  with  two  3D  modules: 
one  module  to  train  automotive  maintenance  tasks  and  one  to  train  elec¬ 
tronics  maintenance  tasks.  The  automotive  versions  were  evaluated  at  the 
Army  Ordnance  Center  and  School  at  Aberdeen  Proving  Ground,  Maryland  and 
the  electronics  versions  were  evaluated  at  the  Army  Air  Defense  School  at 
Fort  Bliss,  Texas. 


Table  1 


AMTESS  Breadboards 


Seville/Burtek 
Microprocessor 
(LSI-11,  16  bit) 


Grumman 

Microprocessor 
(M68000,  16  Bit) 


CRT  (TECT)  and 
Response  Panel 


Color  CRT  with  Touch 
Screen  (Text) 


Rear-Projection 
(35mm  Slides) 


Color  CRT  Pictures 
(from  Video  Disc) 


Diesel  Engine  Mock-Up 


Engine  Components 


Instructor  Station 


Instructor  Station 


Field  Research 

The  field  research  described  below  consisted  of  training  effectiveness 
evaluations  of  the  AMTESS  breadboard  devices  and  analytical  evaluations  of 
the  AMTESS  program.  Because  of  differences  in  device  designs  and  programs 
of  instruction,  we  could  not  directly  compare  the  training  effectiveness 
of  the  breadboards.  Instead,  the  training  effectiveness  of  each  device 
was  compared  to  that  of  conventional  training  (classroom  instruction  sup¬ 
plemented  by  hands  on  practice  with  operational  equipment).  The  analytical 
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is  reported  in  the  literature  is  applied  research.  This  means  that  the 
information  is  usually  based  on  the  comparison  of  entire  devices  to  actual 
equipment  rather  than  the  investigation  of  specific  features.  Therefore, 
the  development  and  use  of  this  prototype  decision  support  system  is  fo¬ 
cusing  the  exisitng  empirically  based  knowledge  of  training  device  con¬ 
figuration  for  efficient  use.  This  focusing  identifies  gaps  in  what  is 
known  about  the  complex  factor  relationships  in  training  device  design  and 
use.  Obviously  research  is  needed  to  close  the  gaps  in  existing  knowledge 
and  also  improve  the  decision  support  system.  We  anticipate  a  productive 
cycle  of  organizing  and  using  empirical  information  to  improve  the  deci¬ 
sion  support  system,  and  identifying  fruitful  areas  for  research  through 
our  efforts  to  improve  and  extend  the  system. 

Summary  of  Handling  Empirical  Data  and  Delivering  Empirically  Based 
Decision  Support 

Our  goal  is  to  provide  decision  support  to  the  Army  in  developing  and 
procuring  cost  effective  and  training  efficient  simulators  and  training 
devices.  We  hope  that  by  providing  empirically  based  decision  support  for 
training  device  specification  and  use,  we  will  also  improve  Army  training. 
We  believe  that  identifying  what  is  known  and  what  is  not  will  also  im¬ 
prove  our  research  efforts  in  training  device  design  and  use.  In  addition 
we  hope  to  be  able  to  integrate  the  information  and  decision  support  that 
this  system  will  be  able  to  provide  with  the  other  automated  systems  (Man 
Integrated  Systems  Technology,  Weddle,  1983;  Early  Training  Estimation 
System,  O'Brien  &  Livingstone,  1980)  that  are  being  devised  to  support  the 
Life  Cycle  System  Management  Model  (see  Rhode,  Skinner,  Mullin,  Friedman, 
Frances,  &  Carroll,  1980  or  Kane  &  Holman,  1982,  for  review)  which  is  used 
by  the  Army  to  manage  acquisition  and  use  of  training  equipment.  The 
previous  section  has  described  our  efforts  to  organize  training  system  and 
training  device  factors  and  to  use  this  organization  to  develop  a  data 
base  on  training  device  information.  It  also  discussed  plans  to  apply 
three  techniques  to  effectively  use  the  information  contained  in  the  data 
base:  meta-analysis,  delphi  analysis,  and  structured  interviews.  Finally, 
plans  for  delivering  this  information  to  the  user  through  a  decision  sup¬ 
port  system  using  expert  system  techniques  were  discussed. 
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Conclusion 


This  presentation  has  described  two  thrusts  of  our  training  device 
research  program.  The  first  thrust  is  to  develop  empirical  data  on  train¬ 
ing  device  design  and  use.  The  second  thrust  is  to  organize  these  data  in 
a  manner  that  will  provide  useful  information  and  guidance  to  the  Army 
training  community.  The  data  generation  effort  began  in  support  of  the  PM 
TRADE  AMTESS  effort.  Some  valuable  data  have  been  developed  through  the 
field  evaluations  and  the  analytical  evaluations  of  the  AMTESS  project. 

We  anticipate  a  series  of  experiments  using  the  AMTESS  devices  to  address 
fundamental  questions  about  a  wide  range  of  training  system  variables.  In 
addition,  the  basic  research  program  on  simulator  fidelity  will  provide 
data  on  a  wide  range  of  fidelity  effects  and  interactions.  These  efforts 
and  other  research  efforts  reported  in  the  literature  will  provide  an 
empirical  base  for  guidance  and  decision  support. 

The  data  handling  and  delivery  of  decision  support  efforts  are  already 
showing  positive  benefits  by  providing  guidance  to  the  data  generation 
research  efforts.  The  organization  of  empirical  information  should  help 
organize  the  entire  field  of  training  device  research,  by  providing  re¬ 
searchers  with  a  solid  compendium  of  the  research  conducted  over  the  last 
thirty  years.  The  development  of  a  data  base  and  the  application  of 
analysis  techniques  futher  contribute  to  the  organization  of  knowledge  to 
support  both  future  research  and  current  decision  support  system  develop¬ 
ment  efforts.  The  use  of  computer  technology  in  developing  a  training 
device  decision  support  system  has  as  a  partial  product  the  advancement  of 
computer  applications  technology.  More  importantly,  a  training  device 
decision  support  system  thoroughly  based  in  empirical  data  will  provide 
an  easily  useable  and  efficient  source  of  information  and  guidance  to  the 
Army  training  community. 
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Appendix 


General  Information  about  Training 
Devices  and  Features 

Typical  Features  of  Simulators  and  Training  Devices 

Fidelity  -  There  are  many  ways  to  talk  about  how  the  training  equip¬ 
ment  approximates  the  actual  equipment,  system  or  environment  that  is 
being  modeled  for  investigation  or  training.  The  term  fidelity  (which 
some  people  will  not  use)  can  be  used  to  describe  the  trainee's  percep¬ 
tions  of  the  equipment,  or  it  can  be  used  solely  to  describe  the  actual 
characteristics  of  the  training  equipment,  or  it  can  be  used  in  several 
other  ways. 

The  US  Army  currently  evaluates  training  devices  and  simulators  along 
two  fidelity  dimensions:  physical  and  functional  (see  TRADOC  Training 
Device  Requirements  Document  Guide,  1979).  Physical  fidelity  refers  to 
the  degree  to  which  the  training  device  or  simulator  matches  the  actual 
equipment.  Functional  fidelity  refers  to  the  degree  to  which  the  training 
device  or  simulator  matches  the  operations  that  are  possible  and  the  in¬ 
formation  presented  in  the  operation  of  the  actual  equipment. 

The  features  which  are  added  onto  the  training  device  or  simulator  can 
thus  degrade  the  fidelity  while  enhancing  the  effectiveness  or  efficiency 
of  training.  Of  course,  the  degree  of  fidelity  can  also  be  lowered  simply 
because  a  high  level  is  not  required  for  training  in  some  particular 
situation.  The  appropriate  level  of  fidelity  or  actual  equipment  repre¬ 
sentation  is  thus  dependent  on  the  type  of  task  being  trained,  the  profi¬ 
ciency  level  desired  as  an  outcome  of  the  training,  the  type  of  overall 
training  program  being  used,  and  the  necessary  adjunct  training  features 
of  the  training  device  or  simulator. 

TRAINING  TYPES 

INDOCTRINATION  -  occurs  early  in  learning  process  (ideally),  covers  what 
the  task  is  and  how  (in  general)  the  task  is  performed 

PROCEDURAL  -  provides  essential  nomenclature  and  sequencing  knowl¬ 

edge  for  task  elements 

FAMILIARIZATION  -  allows  initial  practice  of  task  procedures  and  initial 
learning  of  task/skill  dynamics. 

SKILL  -  allows  practice  toward  developing  proficiency  in  the 

task. 
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TRANSITION  -  practice  that  allows  adaptation  of  previously  acquired 

skills  on  differing  equipment  to  the  new  target  equip¬ 
ment  or  task 

CRISIS  -  training  aimed  at  ensuring  optimal  responses  in  minimal 

or  adequate  time  under  stressful  conditions. 


ADJUNCT  FEATURES 

Freeze  Capability.  Under  certain  conditions  such  as  trainee  error,  the 
device  can  freeze  an  instructional  sequence.  This  allows  evaluation  and 
discussion  of  the  action  sequence  "on  the  fly"  so  that  sub-optimal  action 
patterns  can  be  eliminated. 

Restart/Resequence  Capability.  The  capability  to  restart  an  instructional 
sequence  at  any  point.  This  allows  a  whole-task  trainer  to  be  a  part-task 
or  partial  sequence  trainer  as  well. 

Malfunction  Selection.  Provides  simulated  malfunctions  chosen  by  the 
instructor.  This  allows  training  and  practice  on  the  higher  level  problem 
solving  tasks  associated  with  repair.  At  the  most  complex,  the  system  may 
incorporate  multiple  faults  and  reflect  repair  errors  made  during  train¬ 
ing. 

Sign-in  Capability.  Trainee  can  sign  into  the  device  at  any  authorized 
time  after  providing  specified  information  (passwords,  etc.)  to  the  de¬ 
vice.  This  is  a  valuable  feature  for  individual  record  keeping  or 
self-paced  instruction  when  the  training  device  is  required  to  maintain 
trainee  records. 

Number /Quality  of  Responses.  The  device  can  record,  save,  and  display 
both  the  quality  and  quantity  of  trainee  responses.  This  can  be  a  single 
session  system  that  does  not  maintain  records  between  training  sessions, 
or  it  can  be  a  full  computer-based  tracking  system  that  maintains  total 
training  program  records. 

Internal  Monitoring  of  Instructional  Features.  The  device  can  monitor 
specified  variables  and/or  responses  for  specific  actions  (e.g.,  device 
freezes  if  designated  monitor  reads  in  upper  half  of  scale  or  device  be¬ 
gins  providing  altered  feedback  if  a  designated  control  is  activated). 

This  feature  can  be  used  in  conjunction  with  computer-based  instruction  to 
control  or  branch  to  different  training  problem  presentations. 

Augmented  Feedback.  Under  specific  conditions  or  schedules,  the  device 
can  enhance  the  feedback  received  by  the  trainee.  For  example,  if  the 
information  that  is  used  to  stop,  start,  or  maintain  and  adjust  something 
during  a  task  is  difficult  to  detect  or  observe,  then  this  information  is 
made  more  noticable  in  some  way. 


Supplemental  Information.  In  some  instances,  the  normal  information  that 
is  used  to  guide  a  behavior  cannot  be  altered,  and  therefore  some  addi¬ 
tional  information  is  used  to  ease  training  by  emphasizing  the  Information 
content  in  the  task. 

Mext  Activity  Features.  Introduction  of  the  next  activity  or  lesson  can 
be  linked  to  specific  trainee  actions  by  the  instructor.  This  is  similar 
to  the  branching  activities  used  in  computer-based  training. 

Stimulus  Instructional  Features.  The  instructor/course  developer  can 
specify  the  rates  and  characteristics  of  stimuli  presented  to  the  trainee. 
This  is  basically  a  lower  level  method  for  enhancing  feedback  or  providing 
supplemental  information.  Often  by  eliminating  some  stimuli,  the  salience 
or  importance  of  other  stimuli  is  made  more  obvious.  An  example  would  be 
to  turn  off  or  shut  down  irrelevant  portions  of  the  training  device  during 
certain  training  sequences. 

Automated  Demonstration.  The  capability  to  develop  and  present 
preprogrammed  scenarios  for  trainee  observation  and  instruction  can  pro¬ 
vide  the  trainee  with  a  model  of  expected  performance  or  a  demonstration 
of  the  consequences  of  some  critical  action. 

Record/Playback.  A  demonstration  technique  that  may  be  used  to  record  and 
later  replay  some  portion  of  trainee's  behavior  during  the  training  ses¬ 
sion.  This  can  be  used  to  review  critical  errors  or  review  and  reward 
good  performance,  as  well  as  develop  automated  demonstrations. 

Adaptive  Syllabus.  Techniques  for  computer  control  of  trainee  progression 
based  upon  trainee's  performance.  The  training  scenario  is  varied  as 
performance  changes,  presenting  the  trainee  with  new  knowledge  and  new 
tasks  as  the  old  material  is  mastered  to  some  criteria.  The  most  complete 
level  of  computer-based  instruction  also  allows  for  a  lesson  authoring 
system. 

Lesson  Authoring  System.  Capability  that  allows  for  the  instructor  to 
prepare  lesson  scenarios  or  change  malfunction  and  response  patterns  in 
the  training  program  sequence. 

Record  Keeping  and  Processing.  Ability  to  collect  long-term  performance 
data  on  trainee  and  do  simple  individual,  group,  class  analysis  of  per¬ 
formance  and  develop  ratings  of  progress  through  the  training  program. 
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